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ABSTRACT 


We  study  the  interaction  between  individuals  in  a  population,  where  each  individual  en¬ 
counters  one  another  at  random  intervals,  and  in  each  encounter  the  two  individuals  play 
one  round  of  the  game  of  prisoner’s  dilemma.  By  discounting  future  reward,  and  allowing 
for  imperfect  memory  and  mobility  of  the  individuals,  we  study  the  evolutionary  equilib¬ 
rium  strategy  to  identify  situations  where  cooperation  emerges.  We  find  that  cooperation 
among  individuals  typically  emerges  when  future  reward  becomes  more  important,  when 
individuals  in  the  population  have  better  memory,  and  when  the  individuals  move  in  and 
out  of  the  population  less  frequently.  The  findings  help  explain  social  loafing  and  free  rider 
commonly  seen  in  towns,  corporations,  and  military  units. 
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Executive  Summary 


People  in  a  group  typically  become  less  motivated  to  exert  effort  for  the  common  good, 
when  the  group  size  grows.  This  phenomenon  is  known  as  social  loafing  in  social  psychol¬ 
ogy,  and  there  has  been  extensive  research  on  its  causes  and  remedies.  The  purpose  of  this 
thesis  is  to  use  a  mathematical  model  to  explain  social  loafing.  The  mathematical  model 
uses  the  game  of  prisoner’s  dilemma  as  the  building  block,  and  identifies  situations  when 
cooperation  is  likely  to  emerge.  The  findings  also  suggest  a  few  methods  that  help  motivate 
cooperation. 

Specifically,  this  thesis  presents  a  game-theoretic  model  for  interaction  between  individu¬ 
als  in  a  finite  population.  Each  individual  encounters  one  another  at  random  intervals,  and 
in  each  encounter  the  two  individuals  play  one  round  of  the  game  of  prisoner’s  dilemma. 
The  game  of  prisoner’s  dilemma  has  been  the  canonical  example  in  game  theory,  where 
the  individually  optimal  policy  (defect)  is  opposite  to  the  socially  optimal  policy  (cooper¬ 
ate).  In  other  words,  a  selfish,  rational  player’s  optimal  action  will  lead  to  a  less  desirable 
outcome  for  the  entire  group.  The  prisoner’s  dilemma  has  seen  many  applications  in  arm 
races,  harvest  of  natural  resources,  price  competition  between  retail  stores,  among  others. 
In  our  context,  if  the  two  individuals  in  the  population  encounter  only  once  and  will  not 
see  each  other  again  in  the  future,  then  the  individually  optimal  policy  is  to  defect  (exert 
minimal  effort)  rather  than  to  cooperate  (exert  maximal  effort). 

To  study  the  situations  where  cooperation  is  likely  to  emerge,  we  introduce  three  factors 
into  the  model.  First,  we  introduce  a  discount  factor  so  that  the  utility  earned  in  the  future 
is  worth  less  than  the  same  utility  earned  today.  Second,  we  introduce  the  memory  factor 
of  an  individual,  so  that  each  individual  will  remember  past  encounters  for  some  random 
amount  of  time.  If  an  individual  encounters  another  person  repeatedly,  then  he  can  use  past 
experience  to  decide  what  to  do  in  the  future.  Third,  we  allow  individuals  to  move  in  and 
out  of  the  population  at  random  intervals. 

To  make  our  game-theoretic  model  mathematically  tractable,  we  restrict  each  individual  to 
four  strategies:  tit-for-two-tats  (TF2T),  tit-for-tat  (TFT),  suspicious-tit-for-tat  (STFT),  and 
always  defect  (AD).  The  first  three  strategies  are  similar,  with  the  difference  being  how 


XV 


to  play  a  stranger.  When  encountering  a  stranger,  TF2T  will  begin  by  cooperation  twice, 
TFT  by  cooperating  once,  and  STFT  by  defecting  once.  After  the  initial  moves,  in  each 
encounter,  the  three  strategies  will  mirror  what  the  opponent  did  in  the  previous  encounter. 
The  fourth  strategy  considered,  AD,  simply  defects  every  time.  Therefore,  TF2T  can  be 
viewed  as  the  most  cooperative  strategy,  TFT  the  second  most  cooperative,  while  the  AD 
the  least  cooperative. 

To  analyze  system  equilibrium,  we  focus  on  finding  the  evolutionary  equilibrium  strategy 
for  a  given  set  of  model  parameters.  A  strategy  is  evolutionary  stable — possibly  a  mixed 
strategy — if  it  is  optimal  for  an  individual  to  adopt  the  same  strategy,  when  the  vast  ma¬ 
jority  of  the  population  is  playing  that  strategy.  Depending  on  the  model  parameters,  the 
evolutionarily  stable  strategy  contains  different  subsets  of  the  four  strategies  under  consid¬ 
eration.  We  find  that  cooperation  among  individuals  typically  emerges  (TF2T  and  TFT) 
when  future  reward  becomes  more  important,  when  individuals  in  the  population  have  bet¬ 
ter  memory,  and  when  the  individuals  move  in  and  out  of  the  population  less  frequently. 
We  also  find  that  suspicion  (STFT)  is  a  form  of  behavior  that  increases  the  robustness  of 
cooperation  under  certain  situations. 

In  this  thesis,  several  assumptions  may  not  be  entirely  realistic.  First,  each  player  has 
to  pick  a  strategy  from  a  set  of  four  options,  whereas  in  real-life  people  may  be  more 
creative  in  choosing  a  strategy.  Second,  we  use  the  geometric  distribution  to  model  an 
individual’s  memory,  and  also  for  the  duration  of  time  an  individual  stays  in  the  population 
before  moving  out.  Third,  we  assume  that  the  population  is  homogeneous;  that  is,  every 
individual  has  the  same  value  functions.  In  real-life,  however,  the  pay-off  for  cooperation 
may  be  higher  for  some  people  than  for  some  other  people.  Fourth,  we  assume  that  each 
individual  will  encounter  every  other  individual  in  the  population  with  the  same  probability, 
whereas  in  real-life  neighbors  may  encounter  more  frequently.  Relaxing  these  assumptions 
presents  challenging  future  research  directions. 
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CHAPTER  1: 
Introduction 


1.1  Background 

Social  scientists  use  the  term  “social  loafing”  to  describe  the  reduction  in  cooperation  in  a 
group,  as  the  size  of  the  group  increases.  This  phenomenon  is  surely  familiar  to  anyone  who 
has  worked  on  projects  in  an  academic,  commercial  or  military  setting.  It  is  the  goal  of  this 
study  to  give  a  game  theoretic  explanation  to  the  lack  of  cooperation  in  large  populations. 

The  prisoner’s  dilemma  (PD)  has  been  the  canonical  example  in  game  theory  for  a  game 
where  mutual  defection  is  the  only  equilibrium,  while  it  is  not  in  the  best  interest  of  both 
players  as  a  group.  It  was  introduced  by  Merril  Flood  and  Melvin  Dresher  in  1950,  and  later 
revised  to  the  modern  form  by  Albert  Tucker  (Kuhn,  2009).  In  this  game,  two  players  have 
been  arrested  by  the  police  as  suspects  in  a  joint  crime.  Both  suspects  are  under  pressure 
from  police  investigators  to  confess,  since  they  cannot  produce  enough  evidence  to  convict 
the  two  of  the  crime.  Should  no  one  confess,  they  both  can  expect  to  receive  short  prison 
sentences  for  some  reduced  crime.  Should  one  confess,  while  the  other  holds  out,  the  one 
that  confessed  will  go  free,  while  the  other  bears  the  maximum  prison  sentence.  If  both 
confess,  they  are  both  convicted  and  spend  a  significant  amount  of  time  in  prison,  but  less 
than  the  maximum. 

We  use  standard  notation  for  PD  in  this  thesis.  We  will  call  T  the  pay-off  for  Temptation,  or 
free-riding,  R  the  Reward  for  mutual  cooperation,  P  the  Punishment  for  mutual  defection 
and  S  the  Sucker’s  pay-off.  These  satisfy  the  inequalities  T  >  R  >  P  >  S  and  R  > 

Table  1  presents  the  game  in  standard  matrix  form,  where  C  notates  cooperation,  which  in 
the  case  of  the  prisoners,  is  to  not  confess,  and  D,  defection,  which  for  the  prisoners  is  to 
confess. 

As  seen  in  Table  1.1,  the  game  is  symmetric  between  the  two  players,  and  D  dominates  C, 
since  T  >  R  and  P>  S.  Hence,  the  only  Nash  equilibrium  in  a  single  round  of  PD  is  for  both 
players  to  play  D,  getting  pay-off  P.  However,  both  players  can  do  better  by  both  playing 
C  to  get  pay-off  R.  The  players’  pursuit  of  their  individual  optimum  in  a  single  game 
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C 

D 

c 

{R,R) 

(S,T) 

D 

(T,S) 

(P.P) 

Table  1.1:  Pay-offs  for  a  two-person  prisoner’s  dilemma. 


leads  to  the  Nash  equilibrium  of  both  players  defecting,  which  is  worse  than  the  social 
optimum  of  both  players  cooperating.  This  phenomenon  has  a  wide  range  of  applications 
in  human  interactions,  where  pursuit  of  individual  interests  leads  to  sub-optimal  results  for 
society.  Examples  include  arms  races  between  countries,  harvest  of  natural  resources,  price 
competition  between  retail  stores,  among  others. 

1.2  Literature  Review 

Iterated  prisoner’s  dilemma  (IPD)  is  a  long  term  game  consisting  of  several  rounds,  where 
each  round  is  a  PD.  The  game  is  a  subject  of  research,  as  it  allows  a  socially  optimal  equi¬ 
librium  to  emerge  under  certain  conditions.  IPD  has  been  studied  extensively  theoretically 
and  experimentally.  Axelrod  (1984)  has  presented  a  theoretical  framework  and  a  series  of 
experiments  conducted  in  the  form  of  competitions  between  computer  algorithms.  He  finds 
that  cooperation  will  be  beneficial  if  “the  future”  is  important  enough.  Axelrod  continues 
to  define  and  find  the  value  of  the  “shadow  of  the  future”  parameter,  also  called  the  dis¬ 
count  parameter,  for  which  the  strategy  of  tit-for-tat  is  collectively  stable.  Tit-for-tat  is  a 
strategy  the  begins  by  cooperating,  and  continues  by  mirroring  the  other  player’s  moves,  in 
all  subsequent  rounds. 


acriticai  =  max 


T-R  T-R\ 
T-P^  R-S)^ 


where  a  is  the  discount  parameter. 

Cooperation  in  groups  has  been  researched  as  an  iterated  game  based  on  PD.  Researchers 
use  the  concept  of  N-person  IPD  (NIPD).  In  this  game  N  players  are  engaged  in  an  IPD. 
Each  round  of  the  game  all  players  play  simultaneously.  It  may  be  parametrized  with  b/N 
as  the  reward  to  each  player  for  every  player  that  cooperates  and  c  as  the  personal  cost  of 
cooperation  for  each  player.  If  all  cooperate  then  each  member  will  gain 
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bN 

-^-c-b-c 

It  is  assumed  that  b  >  c  >  ^,  m  order  to  maintain  the  assumptions  of  PD  (Bendor  & 
Mookherjee,  1987).  Cooperation  between  all  is  possible  under  certain  conditions  using 
different  strategies.  Research  has  been  conducted  to  ascertain  limits  to  this  cooperation  in 
groups. 

Joshi  (1987)  studies  tit-for-tat  strategies  in  NIPD.  Several  strategies  were  explored  with 
varying  degrees  of  leniency  towards  defection.  Joshi  characterizes  the  strategies  according 
to  the  proportion  of  cooperators  in  the  group  needed  for  a  certain  player  to  cooperate.  The 
least  lenient  of  these,  the  strategy  requiring  total  cooperation  by  everyone  in  the  group 
for  continued  collaboration  he  dubs  “hard”  and  finds  it  to  be  stable  against  defection.  It 
was  found,  however,  that  as  the  number  of  players  in  the  game  increased,  the  threshold 
for  the  frequency  of  the  “hard  TFT”  strategy  to  become  dominant  rose.  This  suggests  that 
evolution  of  cooperation  in  this  scenario  is  more  difficult  in  larger  groups. 

Bendor  &  Mookherjee  (1987)  find  that  in  NIPD  “If  relative  to  group  size  n  the  punishment 
phase  is  sufficiently  long  and  the  members  do  not  discount  the  future  very  heavily,  there  is 
an  equilibrium  in  which  all  members  cooperate  every  period.  However,  given  the  discount 
rate  a  there  is  an  upper  limit  n*  beyond  which  cooperative  outcome  cannot  be  upheld.” 
They  find  the  value  for  the  critical  group  size 


n 


* 


ab  —  b 
ab  —  c^ 


where  b/n  is  the  contribution  of  cooperation  by  any  player  to  the  pay-off  of  others  and  c 
is  the  personal  cost  of  cooperating.  In  a  study  of  common  goods  problems  with  various 
utility  functions,  Pecorino  (1999),  finds  similar  results  for  cooperation  being  feasible,  yet 
increasingly  hard  to  achieve  as  the  number  of  players  in  a  game  grows. 

A  different  approach  towards  studying  iterated  games  in  large  populations,  is  to  treat  a 
grid  of  players  each  interacting  only  with  his  neighbours,  called  spatial  IPD.  Applying  this 
approach  has  led  Nowak  &  Jay  (1993)  to  conclude  that  the  dynamics  of  such  games  are 
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significantly  different  than  homogeneous  IPD  and  may  be  chaotic. 

“Social  loafing”  is  a  term  coined  by  social  psychologists  as  the  deterioration  of  cooperation 
as  group  size  increases  (Karau  &  Williams,  1993).  We  have  described  two  game-theoretic 
approaches,  the  N-person  iterated  prisoner’s  dilemma  and  the  spatial  IPD,  that  may  offer 
explanations  for  this  type  of  observational  research. 

1.3  Contribution 

In  this  thesis  a  model  of  iterated  games  in  finite  populations  is  used  to  analyze  dynamics  of 
large  populations.  The  model  allows  for  specific  targeting  of  punishment  against  defectors 
and  may  allow  a  more  accurate  prediction  of  conditions  under  which  cooperation  is  benefi¬ 
cial.  This  model  may  offer  an  appropriate  explanation  to  situations  of  cooperation  in  large 
groups.  Specifically,  we  make  the  following  contribution  to  the  literature. 

1 .  Develop  a  game-theoretic  model  for  individuals  who  meet  randomly  in  a  population, 
where  in  each  encounter  the  two  individuals  interact  according  to  PD. 

2.  For  a  given  set  of  model  parameters,  develop  a  method  to  mathematically  identify 
the  equilibrium  states  of  the  game-theoretic  model. 

3.  Explain  the  influence  of  several  factors  on  the  feasibility  of  cooperation.  These  fac¬ 
tors  include  (1)  population  size,  (2)  memory  and  mobility  of  individuals  in  the  pop¬ 
ulation,  and  (3)  discount  factor. 

4.  Explain  the  importance  of  suspicion  to  the  robustness  of  cooperation. 
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CHAPTER  2: 
The  Model 


Consider  a  finite  population  with  N  individuals.  The  game  proceeds  one  round  at  a  time, 
and  goes  on  indefinitely.  In  each  round,  every  member  of  the  population  is  randomly 
paired  with  another  member  for  a  single  round  of  prisoner’s  dilemma.  In  other  words,  each 
member  in  the  population  faces  a  steady  stream  of  members  who  are  randomly  chosen  from 
a  fixed  population  with  replacement.  In  a  small  population,  a  player  may  expect  to  see  the 
same  players  again  and  again  at  small  intervals,  while  for  larger  populations  a  player  may 
expect  to  meet  players  few  times  in  the  foreseeable  future.  For  each  encounter  between  two 
players  a  standard  prisoner’s  dilemma  (PD)  is  played  with  the  pay-off  shown  in  Table  1. 

In  this  model,  the  concept  of  a  “foreseeable  future”  is  governed  by  two  factors:  a  discount 
factor  a  and  a  memory  factor  j8 .  The  discount  factor  a  signifies  that  a  reward  1  received 
in  the  next  round  is  only  worth  a  as  in  the  present  round.  The  memory  factor  models 
how  well  each  individual  remembers  encounters  with  the  other  individuals  from  the  past. 
Specifically,  if  two  players  recognize  each  other  at  the  current  round,  then  they  will  rec¬ 
ognize  each  other  in  the  next  round  with  probability  j8,  or  forget  about  each  other  in  the 
next  round  with  probability  1  —  /3 .  In  other  words,  the  number  of  turns  before  memory 
of  an  encounter  between  two  players  is  lost  is  a  geometric  random  variable.  The  memory 
factor  P  can  also  be  interpreted  as  a  mobility  factor.  In  each  round,  with  probability  j8,  an 
individual  in  the  population  may  move  out  and  be  replaced  by  another  individual  moving 
in. 

The  model  is  governed  by  seven  parameters,  the  four  standard  parameters  of  a  PD,  two 
parameters  for  the  foreseeable  future  and  a  single  parameter  for  the  population  size.  The 
size  of  the  population  will  be  referred  to  as  N,  future  discount  as  a,  memory  retention  as  /3 
and  the  notation  for  pay-offs  in  the  PD  are  T,  R,  P  and  S. 

2.1  Model  Strategies 

The  model  presented  so  far  is  rather  general  and  there  exists  a  myriad  of  player  strategies. 
However,  for  the  purpose  of  studying  the  emergence  and  failure  of  cooperation  we  choose 
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to  introduce  four  simple  strategies.  The  pool  of  four  strategies  will  be  assumed  to  be  the 
only  options  available  to  the  N  players  in  the  population.  Each  player  may  select  a  strategy 
out  of  the  pool  and  play  accordingly  for  the  remainder  of  the  game.  The  four  strategies  are 
tit-for-tat  (TFT),  tit-for-two-tats  (TF2T),  suspicious-tit-for-tat  (STFT)  and  always  defection 
(AD).  This  assumption  limits  the  generality  of  our  results. 

2.1.1  Always  Defection 

A  strategy  that  defects  unconditionally  in  every  round  of  the  game,  as  such  it  is  designated 
mean.  This  is  the  dominant  strategy  for  a  single  round  of  prisoner’s  dilemma. 

2.1.2  Tit-For-Tat 

The  TFT  strategy,  first  presented  by  Anatol  Rapaport  for  PD  computer  tournaments  held  by 
Robert  Axelrod  (Axelrod,  1984),  has  been  studied  extensively  and  shown  to  be  an  effective 
strategy  in  many  situations.  The  strategy  is  extremely  simple.  If  it  meets  an  opponent 
whom  it  has  never  played  before  (or  doesn’t  remember  playing  before),  it  plays  cooperate. 
From  that  point  on  it  mirrors  the  opponent’s  previous  move.  This  strategy  is  a  nice  strategy 
since  it  will  never  defect  first  (Axelrod,  1984).  If  both  players  use  TFT,  then  cooperation 
occurs  with  each  player  receiving  a  stream  of  rewards. 

2.1.3  Suspicious-Tit-For-Tat 

This  strategy  is  a  variation  of  standard  TFT.  The  difference  lies  in  the  strategy’s  action 
against  an  unknown  opponent.  This  strategy  will  begin  an  interaction  by  defecting  and  will 
then  mirrors  the  opponent’s  moves.  This  strategy  may  be  said  to  be  mean  as  it  will  never 
cooperate  first.  STFT  is  important  to  this  paper,  as  we  conjecture  that  use  of  suspicion  by  a 
population  is  a  method  for  increasing  the  robustness  of  cooperation. 

2.1.4  Tit-For-Two-Tats 

This  strategy  was  suggested  by  Robert  Axelrod  for  the  second  of  his  computer  tournaments 
as  a  solution  to  the  problem  of  two  retaliatory  strategies  (such  as  TFT)  entering  a  cycle 
of  alternating  cooperation  and  defection  (Axelrod,  1984).  This  nice  strategy  begins  by 
cooperating  twice  and  then  continues  to  mirror  it’s  opponents  moves.  In  that  case,  it  will 
begin  to  defect  until  the  opponent  cooperates  once. 
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2.2  The  Game  in  Standard  Matrix  Form 

In  order  to  examine  the  dynamics  among  the  four  strategies  in  this  finite-population  model, 
the  first  step  is  to  define  an  objective  function  for  each  member  in  the  population.  In  this 
thesis,  we  assume  that  the  objective  for  each  member  is  to  maximize  his  expected  total 
discounted  utility.  In  other  words,  if  Xj  represents  the  utility  received  in  round  t,  then  a 
member’s  objective  function  is  to  maximize 

oo 

Y,cc'E[Xr].  (2.1) 

t=0 


An  alternative  approach  to  compute  the  objective  function  in  (2.1)  is  to  first  compute  the 
expected  total  discounted  utility  collected  through  one  match-up.  A  match-up  is  a  sequence 
of  encounters  between  two  players  until  the  two  players  forget  about  each  other.  In  other 
words,  if  a  player  meets  someone  whom  he  does  not  recognize  (an  opponent  he  has  never 
played  before,  or  an  opponent  he  has  played  before  but  forgets),  then  a  new  match-up 
begins.  If  a  player  meets  someone  he  recognizes  from  previous  encounters,  then  that  round 
is  part  of  an  ongoing  match-up. 

Let  aij  denote  the  total  expected  discounted  utility  collected  through  one  match-up, 
if  a  member  plays  strategy  i  against  another  member  who  plays  strategy  j,  for  i,j  G 
{TF2T,TFT,STFT,  AD}.  The  matrix  A  is  shown  explicitly  in  the  following. 


TF2T 

TFT 

STFT 

AD 

TF2T 

«1,2 

«1,3 

«l,4^ 

TFT 

«2,1 

«2,2 

«2,3 

«2,4 

STFT 

«3,1 

«3,2 

«3,3 

«3,4 

AD 

\  «4,1 

«4,2 

«4,3 

«4,4  / 

In  this  matrix,  a23,  for  example,  is  the  discounted  total  expected  utility  a  TFT  player  col¬ 
lects  through  a  match-up,  if  he  meets  a  stranger  who  plays  STFT. 


Suppose  the  population  plays  a  mixed  strategy  ,  ^2,  ^3,  ^4)-  That  is,  each  time  a  member 
meets  someone  he  does  not  recognize  (an  opponent  he  has  never  played  before,  or  an  op¬ 
ponent  he  has  played  before  but  forgets),  the  opponent  will  play  strategy  j  with  probability 
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qj,  for  j  =  1,2, 3,4.  By  letting  4  =  1  if  the  member  does  not  recognize  the  opponent  in 
round  t  (beginning  of  a  new  match-up),  and  4  =  0  if  the  member  recognizes  the  opponent 
in  round  t  (part  of  an  ongoing  match-up),  then  an  equivalent  expression  for  the  objective 
function  in  (2.1)  is 

f=0  \  \i=l  /  /  \i=\ 

Since  (L^o  Vt] )  is  a  constant  regardless  the  strategy  played  by  the  member,  to  maxi¬ 

mize  the  preceding  it  is  equivalent  to  choose  i  to  maximize 

4 

Consequently,  an  equivalent  objective  function  for  a  member  is  to  choose  a  strategy  that 
maximizes  the  total  discounted  expected  utility  collected  in  a  match-up.  We  next  explain 
how  to  compute  aij  in  (2.2). 


yt=0 


2.2.1  Match-up  Between  Two  Nice  Strategies 

The  match-up  between  two  nice  players  will  be  repeated  cooperation,  as  both  players  will 
never  defect  first.  At  the  beginning  of  a  round,  denote  the  total  expected  discounted  utility 
collected  in  the  remainder  of  this  match-up  by  h{R),  if  two  players  will  always  play  C,  so 
that  in  each  encounter  a  player  receives  R.  In  the  present  round,  the  two  players  will  meet 
with  probability  1/A.  If  they  meet,  then  each  player  gets  a  reward  R;  otherwise,  there  is 
no  immediate  reward.  With  probability  j8,  the  two  players  will  remember  each  other  in  the 
next  round,  and  will  collect  h{R)  in  the  next  round,  which  is  discounted  by  a.  If  they  do 
not  meet,  with  probability  1  —  1/A,  then  they  will  just  get  h{R)  in  the  next  round,  if  they 
remember  each  other  in  the  next  round.  Therefore,  we  can  set  up  one  equation  involving 
h{R)  as  follows: 


HR)  =  ^(R  +  amR))  +('-/) 
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Solving  for  h{R)  gives 


m 


R 

N{l-al5)' 


The  funetion  h{R)  ean  be  interpreted  as  the  total  expeeted,  diseounted  utility  in  the  remain¬ 
der  of  the  mateh-up,  if  a  player  reeeives  R  from  eaeh  future  encounter.  This  can  then  be 
used  in  the  expected  utility  of  a  match-up  involving  only  TF2T  and  TFT. 


«i,i  =  «2,2  =  «2,i  =  «i,2  =R  +  aj5h{R) 


(2.3) 


2.2.2  Match-up  Between  Two  Mean  Strategies 


The  match-up  between  mean  strategies  will  never  have  cooperation  introduced,  so  the  ex¬ 
pected  utility  is  derived  exactly  like  the  previous  section,  except  for  the  per  round  pay-off 
being  P  instead  of  R.  Replacing  R  in  equation  (2.3)  with  P  gives 


(34,4  =  «3,3  =  «3,4  =  «4,3  =P  +  aj5h{P) 


2.2.3  Match-up  Between  TFT  and  AD 

The  first  encounter  of  TFT  and  AD,  in  a  match-up,  will  see  TFT  getting  the  sucker’s  pay¬ 
off,  while  AD  gets  the  temptation  pay-off.  Every  following  encounter  will  be  mutual  defec¬ 
tion,  resulting  in  an  expected  pay-off  of  h{P).  Similar  to  equation  (2.3),  we  can  conclude 
that 


‘•V-S  +  ctli 
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2.2.4  Match-up  Between  TF2T  and  STFT 


In  the  first  encounter,  TF2T  gets  S,  while  STFT  gets  T .  In  the  second  encounter  and  all 
following  encounters,  both  strategies  will  get  the  steady  stream  7?.  Similar  to  equation  (2.3), 
we  can  conclude  that 


a, ,3  -S  +  alih(R)  -S  +  ap  ■ 

Similarly,  the  pay-off  for  the  STFT  strategy  is 


03,1  - 7-  +  alih(R]  -T  +  ali  ■ 

2.2.5  Match-up  Between  TF2T  and  AD 

In  the  case  of  TF2T  against  the  AD  strategy,  the  TF2T  will  allow  two  defections  against 
it  before  the  interaction  collapses  into  constant  defection  on  both  sides.  We  calculate  by 
conditioning  on  the  number  of  rounds  between  the  first  encounter  and  second  encounter  in 
the  match-up. 


=  5  + 


N-l  -ajS)) 


(^)  (a|3)'o2,4 

~  if) 

1  -  (l  -  ^) 


The  pay-off  for  AD  in  this  match-up  will  be  similar  with  the  exception  of  T  replacing  S. 
That  is. 


^43-7’  +  )^ 


[r  +  a^ 


n) 

1  -  (l  -  f) 
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2.2.6  Match-up  Between  TFT  and  STFT 

A  match-up  between  TFT  and  STFT  will  always  start  by  a  defection  on  the  side  of  the 
suspicious  strategy  and  continue  with  an  endless  cycle  of  defection  on  alternating  sides. 
This  means  that  the  expected  utilities  of  the  two  strategies  in  this  match-up  are  entwined. 
We  use  X  to  mark  the  expected  value  of  an  alternating  T  and  S  match-up  starting  with  the 
sucker’s  pay-off  S,  and  y  for  the  same  match-up  starting  with  the  temptation  pay-off  T .  We 
condition  on  the  number  of  rounds  between  the  first  encounter  and  second  encounter  to  set 
up  two  equations: 


/—I 


y  ajS 


^  =  s+yy-  (ai5y  =  s+^ - ^ 

Solving  the  two  equations  for  x  and  y  leads  to  the  following 


/—I 


ci2yi  —  S  ccpy  —  S  ccp  - 


T{l-al5  +  f)+sf 


ci^  2  —  T  -y-  ocpx  —  T  (jcp  - 


A(l-a/3)(l-aj8+2f)’ 
S(l-a[i  +  f)+Tf 


A(l-aj8)  (l-a/3  +  2^ 


2.2.7  Game  Matrix  in  Normal  Form 

Bringing  together  all  the  results  of  the  previous  section  gives  us  the  following  game  matrix 
in  normal  form. 


TF2T 


^  R  +  aP  \ 

R 

R  +  aP  1 

^  R 

KNil-aP) 

T  +  aP\ 

(  ^ 

[N{l-ap) 

V+  iv(l-a/3(l-‘)) 


T  +  ap 


TFT 

s(l-ap+^)+T'f 


STFT 


S+aP 
S+ap 


R 

N{l-ap) 

r(l-a/3+!^)+5^ 

alS\ 

-w 


mi-al3)[l-al3+2f') 


A'(l-a^)(l-aj3+2 


5  + 


AD 

iv(l-a/S(l-^)) 


S  +  ap 


-ap  ( 


ap  ( 


N{l-ap)^ 


N(l-aP))  j 


(2.4) 
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CHAPTER  3: 
Analysis 


In  this  thesis,  we  wish  to  find  the  connection  between  population  size  and  other  parameters 
of  a  game  and  the  possibility  of  cooperation  in  a  stable  steady  state.  To  achieve  this  goal, 
we  first  define  a  stable  steady  state.  The  common  definition  for  such  a  state  is  the  Nash 
equilibrium.  A  Nash  equilibrium  of  a  game  is  a  profile  of  actions  (strategy),  such  that  each 
player’s  action  is  optimal  against  the  actions  of  the  other  players.  In  other  words,  no  player 
can  increase  his  expected  utility  by  switching  to  a  different  strategy,  given  that  all  the  other 
players  play  their  Nash  equilibrium  strategy  (Osborne  &  Rubinstein,  1994). 

Whereas  Nash  equilibrium  specifies  a  stable  state  involving  several  players,  an  evolution- 
arily  stable  strategy  (ESS)  is  a  refinement  of  Nash  equilibrium  that  applies  to  a  population. 
Specifically,  an  ESS  is  a  (possibly)  mixed  strategy  that,  if  adopted  by  a  population,  can¬ 
not  be  invaded  by  other  strategies  (Smith,  1974).  Write  E\U ^V]  for  the  expected  utility  of 
the  strategy  1/  in  a  match-up  against  the  strategy  V.  We  say  S  is  an  ESS  if  either  of  the 
following  two  conditions  hold: 

1.  E[S,S]  >  E[T,S],  for  all  T  ^  S;  or 

2.  E[S,S]=E[T,S]  andE[S,r]  >  E[r,  T],  for  all  T  ^  5. 

In  this  thesis,  we  seek  to  determine  the  ESS  for  our  game-theoretic  model  introduced  in 
Chapter  2,  as  we  attempt  to  ascertain  the  conditions  required  for  a  long  term  cooperative 
equilibrium  in  a  dynamic  population.  In  this  thesis,  we  focus  on  finding  sets  of  game 
parameters  that  allow  a  cooperative  evolutionary  stable  (mixed)  strategy  to  emerge. 


3.1  Determine  Evolutionary  Stable  Strategy 

In  this  section,  we  analyze  the  two-person  non-zero-sum  game  with  pay-offs  given  in  (2.4), 
to  determine  the  ESS  for  given  parameters.  To  begin,  first  we  recognize  the  following  string 
of  inequalities,  which  will  be  helpful  in  the  following  analysis. 

^31  >  an  =  ai2  =  Cl22  =  (^2l  >  <^33  =  <^34  =  (^44  =  <^43  >  (^24  >  ^14  (3.1) 


13 


We  seek  to  find  the  ESS.  In  general,  the  population  may  be  playing  all  four  strategies  with 

proportions  expressed  in  the  tuple  x.  Let  x  =  (xtf2T,.^tft,.^stft,.^ad),  where  =  I 

iei 

and  I  =  {TF2T,TFT,STFT,  AD}.  This  problem  may  have  at  most  15  evolutionary  stable 
strategies,  including  four  involving  one  pure  strategy,  six  involving  two  pure  strategies,  four 
involving  three  pure  strategies,  and  one  involving  all  four  pure  strategies. 


For  each  subset  7  C  /,  we  want  to  determine  whether  there  exists  an  ESS  that  involves  all 
strategies  in  7  and  none  in  7  \  7.  Write  A  for  the  sub  matrix  of  A  by  removing  rows  and 
columns  not  in  7.  If  there  exists  an  ESS  that  involves  all  strategies  in  7,  then  the  population 
distribution  among  the  strategies,  denoted  by  x,  must  be  chosen  so  that  for  each  strategy 
in  7,  the  expected  utility  against  the  population  distribution  x  must  be  the  same.  Hence, 
a  necessary  condition  for  x  to  constitute  an  ESS  is  for  x  to  satisfy  the  following  equation, 
where  v  is  a  constant. 


(  X,  \ 


/v\ 


Ax  =  A 


(3.2) 


\Xm  J  \  V  y 


The  solution  x  to  the  preceding  equation  is  only  a  necessary  condition  for  x  to  be  an  ESS, 
but  not  a  sufficient  condition.  For  x  to  be  an  ESS,  we  need  to  verify  that  it  will  not  be 
invaded  by  the  strategies  in  any  strategy  i  ^  7.  In  other  words,  we  need  to  verify  that  the 
expected  utility  for  any  strategy  i  ^  7  against  x  is  strictly  less  than  v.  Mathematically,  we 
need  to  verify  that 

Y^aijXj<v  (3.3) 

for  i  xjL  7. 


3.2  ESS  Involving  One  Pure  Strategy 

A  pure  strategy  constitutes  an  ESS,  if  it  is  best  to  play  that  pure  strategy  when  everyone 
else  in  the  population  plays  that  same  pure  strategy.  In  other  words,  a  pure  strategy  is  an 
ESS,  if  and  only  if  a  diagonal  entry  in  the  matrix  A  is  the  largest  in  its  column. 
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3.2.1  AD  Pure  ESS 

The  AD  constitutes  an  ESS  solution  for  this  game,  since  against  a  population  of  players 
always  defecting,  a  single  player  can  do  no  better  than  to  defect  along  with  everyone  else. 


3.2.2  TFT  Pure  ESS 

For  TFT  to  be  a  stable  pure  strategy,  the  pay-off  to  a  player  playing  TFT  needs  to  be  the 
greatest  among  four  strategies,  given  that  everyone  else  is  playing  TFT.  In  other  words,  we 
require 

«2,2  >  «4,2  and  ^2,2  >  «3,2- 

Solving  for  N  gives  two  conditions  for  the  limiting  population,  for  which  cooperation  can 
be  maintained,  as  long  as  either  condition  is  met. 


P  +  R-S-T  >0 
P  +  R-S-T  <0 


and 

and 


a/3  R-P 
^  1-apT-R' 
ap  2R-S-T 
l-ap  T-R 


This  means  that  depending  on  the  sign  of  P  +  R  —  S  —  T,  if  the  population  is  less  than  the 
appropriate  threshold,  everyone  plays  TFT.  However,  if  the  population  should  grow  past 
the  threshold  then  all  players  would  immediately  revert  to  AD. 


3.2.3  STFT  Pure  ESS 

For  STFT  to  be  a  stable  pure  strategy,  the  pay-off  to  a  player  playing  STFT  needs  to  be  the 
greatest  among  four  strategies,  given  that  everyone  else  is  playing  STFT.  In  other  words, 
we  require 

«i,3  <  «3,3  and  ^2,3  <  «3,3- 

Solving  for  N  gives  two  conditions  for  the  limiting  population,  for  which  cooperation  can 
be  maintained,  as  long  as  either  condition  is  met. 


15 


S  +  T-R-P>0  and 
S  +  T 

S  +  T-R-P<0<—^ 


N> 

and 


aj8  T  +  S-2P 
l-aj8  P-S  ’ 
aj8 

> - - - . 

l-ap  P-S 


A  large  population  may  be  all  playing  STFT  in  a  suspicious  ESS.  However,  if  the  population 
decreases  beyond  a  certain  threshold,  the  population  may  be  invaded  by  TF2T  players,  who 
will  begin  cooperation. 


3.2.4  TF2T  Pure  ESS 

TF2T  by  itself  can  never  be  a  ESS  since  the  best  one  player  can  do  against  TF2T  is  STFT, 
as  seen  by  >  a2,i  in  (3.1). 


3.3  ESS  Involving  Two  Pure  Strategies 

In  this  section,  we  use  equation  (3.2)  to  determine  a  necessary  condition  on  the  population 
size  N  for  ESSs  that  involves  two  pure  strategies.  There  are  (2)  =  6  cases.  Recall  that  we 
need  to  use  equation  (3.3)  to  determine  whether  an  ESS  does  exist,  which  will  be  done  in 
Chapter  4. 

3.3.1  TF2T  and  TFT 

For  this  strategy  pair  the  game  matrix  becomes  degenerate  with  all  four  elements  of  A 
being  equal.  In  this  case,  it  is  clear  that  any  distribution  of  the  population  between  the 
two  strategies  is  a  candidate  for  ESS.  For  each  population  distribution,  one  needs  to  use 
equation  (3.3)  to  verify  whether  it  is  indeed  an  ESS. 

3.3.2  STFT  and  AD 

For  this  strategy  pair  the  game  matrix  also  becomes  degenerate  with  all  four  elements  of 
A  being  equal.  Thus  any  distribution  of  the  population  between  the  two  strategies  is  a 
candidate  for  ESS.  For  each  population  distribution,  one  needs  to  use  equation  (3.3)  to 
verify  whether  it  is  indeed  an  ESS. 
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3.3.3  TF2T  and  STFT 

The  game  matrix  for  this  strategy  pair  is 


TF2T 

TF2T/«+a/5(s;^) 
STFT  r  + a/s  (j,^) 


STFT 

S+CCl^  (/V(l^a/3)_ 
P  +  aP  fjv(l-aj8) 


Since  an  <  a2i,  if  a^  <  <322,  then  STFT  dominates  TF2T  In  that  case  an  ESS  does  not 
exist  with  only  these  two  strategies  active.  As  a  necessary  condition  for  an  ESS  for  the 
population  with  TE2T  and  STET  strategies  active,  we  solve  di2  >  <322,  or  equivalently, 


S+aj5 


R 


A(l-aj8) 


>P  +  al5 


N{l-ap) 


Solving  for  a  threshold  population  value  gives  a  necessary  condition,  for  which  an  ESS 
may  exist  as  the  following 


aj5  R-P 
l-ap  P-S' 


(3.4) 


Below  this  threshold  there  may  be  an  ESS  with  some  in  the  population  playing  TE2T  and 
the  rest  playing  STET.  Above  the  threshold  the  entire  population  would  revert  to  playing 
STET,  as  suspicion  will  be  their  best  option.  We  emphasize  that  equation  (3.4)  is  only  a 
necessary  condition  for  an  ESS,  and  one  needs  to  use  equation  (3.3)  to  verify  whether  it  is 
indeed  an  ESS. 


3.3.4  TFT  and  STFT 

The  game  matrix  for  this  strategy  pair  is 


A  = 


TET 


STET 


/ 


V 


R  +  a[5 

^  N{l-ap){l-ap+2f 


STET 

^iV(l-a/3)(l-ai3+2‘f) 


P+aP 


N{l-ap) 


I 
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For  an  ESS  with  both  TFT  and  STFT  to  be  active  at  the  same  time  there  can  be  two  cases: 


L  di  j  >  <32,1  and  <31,2  <  <32, 2-  This  can  be  written  in  the  following  two  inequal¬ 
ities. 


R  +  aP 


R  \  S{l-aP  +  f]+T^ 

>T  +  ap 


N{l-al5) 


and 


P+aj5 


>  S+aj5 


iV(l-aj8)(l-a/3+2^ 

T(l-al5  +  f)+sf 


These  can  be  simplified  to  give  the  following  necessary  condition  for  ESS. 


5_1_7’ 

T  +  S  —  R<P<  — - —  and 


aB  T  +  S-2P  aB  2R-T-S 

<N< 


S  +  T 

P  >  — - —  and  N  < 


l-aj8  P-S  "  l-aj8  T-R  ’ 

a/3  2R-T-S 


l-aj8  T-R 


2.  <3i,i  <  <32,1  and  di.2  >  <32.2-  These  are  the  same  inequalities,  with  the  inequal¬ 
ity  signs  reversed.  This  is  simplified  to  find  the  following  necessary  condition  on  an 
ESS. 


P<T+S-R 


aB  2R-S-T  aB  T  +  S-2P 

- - P-S  ■ 


l-aj8  T-R 


3.3.5  TF2T  and  AD 


The  game  matrix  for  this  strategy  pair  is 


A  = 


TF2T 

AD 


/ 

V 


TF2T 

rp  I  N(l^aP}} 


AD 


f,  I  N(l^ali))  \ 

A^(l-a/3(l-i)) 

(lV(l^ai3))  y 


Since  di2  <  <322>  if  <3ii  <  d2i  also  holds,  then  AD  dominates  TF2T.  Therefore,  a  mixed 
strategy  ESS  is  possible  only  if  dn  >  a2i,  or  equivalently. 
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R  +  aP 


R 


N{l-al5) 


ap  (T  +  aP 


N{l-aP) 


Nil-aHil-i;))  ■ 


Solving  for  a  threshold  population  value  gives  a  necessary  condition  that  an  ESS  may  exist. 


N< 


aj5 

l-aj5 


In  this  case  an  ESS  may  exist  with  some  playing  AD,  while  the  rest  play  TE2T.  Above  the 
threshold,  the  entire  population  reverts  to  playing  AD. 


3.3.6  TFT  and  AD 

The  game  matrix  for  this  strategy  pair  is 

TET  AD 

^  ^  TET  /  R  +  aj5 S+aj5  yv(i_«/3)  \ 

AD  +  P+(^P{n{T^)) 

Since  di2  <  ^22,  an  ESS  with  both  strategies  may  exist  only  when  an  >  <321,  which  results 
in  the  following  condition  for  the  population  size. 

aj8  R-P 

N  < - - - . 

1-apT-R 

Similar  to  the  previous  sections,  this  equation  gives  a  threshold  below  which  cooperation 
is  preferred  to  defection.  We  may  expect  the  population  to  have  an  ESS  in  which  at  least 
a  certain  proportion  are  cooperating  by  using  TET.  Above  the  threshold,  all  will  revert  to 
playing  AD. 
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3.4  ESS  Involving  Three  Pure  Strategies 


The  full  game  matrix  A  in  (2.4)  contains  two  2x2  sub-matrices  along  the  main  diagonal 
whose  elements  are  all  equal.  Removing  any  strategy  from  the  game  will  leave  a  3x3 
matrix,  which  we  call  A.  The  sub-matrix  A  contains  at  least  one  of  the  two  degenerate 
sub-matrices.  A  three-strategy  ESS  can  be  represented  by  probability  vector.?  G  whose 
sum  is  1.  According  to  equation  (3.2),  the  probabilities  must  satisfy  the  condition  that 


Ax  = 


/  v\ 

V 

V  v/ 


(3.5) 


for  some  v  G  M.  In  particular,  the  two  rows  that  contain  the  degenerate  2x2  sub-matrix 
must  be  equal  to  each  other  when  weighted  by  x.  Consider  an  example  by  removing  the 
AD  strategy. 


TF2T 

/  R  +  aP  ^  ^ 


A  = 


N{l-aP) 

R  +  aP‘  ^ 


TFT 

R  +  aP'  ^ 


Nii-aP) 

T  +  ap'  ^ 


^N{l-ap) 

R  +  aP'  ^ 


T  I  cifl 

^Af(l-a/3)(l-ai3+2‘f) 


STFT 

S+aP  ^ 

S  +  aP 


iV(l-a/3)(l-a/3+2^) 

P  +  ap^  ^ 


^  ^  ^  V^l-a/D 

The  equality  between  the  first  two  rows  of  the  condition  expressed  in  (3.5)  is  then 


/ 


ildi  j  +.?2dl,2  +-?3«1,3  =  -?1«2,1  +X2a2,2  +-?3«2,3- 


But  since  dyi  =  d  1,2  =  <32,1  =  <32.2,  the  preceding  simplifies  to 


dl,3  =  d2,3- 

The  result  is  simply  an  equality  that  can  be  solved  for  A  as  a  function  of  a,  ,  r,R,  F,  S.  An 
ESS  with  any  three  of  the  strategies,  at  best,  may  only  exist  for  a  precise  population  size. 
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making  it  extremely  unstable  with  relation  to  population  changes. 


3.5  ESS  Involving  Four  Pure  Strategies 

An  ESS  with  all  four  strategies  active  at  the  same  time  may  exist  only  if  there  is  no  dom¬ 
inance  between  the  rows  in  matrix  A.  Due  to  the  degeneracy  in  the  two  2x2  sub-matrices 
and  the  fact  that  a2,4  >  ai  4,  one  of  the  following  must  hold  for  there  to  be  no  dominance 
between  rows. 


1.  a  1,3  >  a2,3  and  03^2  >  «4.2  and  >  «3,i-  These  inequalities  do  not  result  in  any 
feasible  solutions  for  N. 

2.  >  a23  and  a3^2  <  «4,2  anda4,i  <  ^3.1.  With  some  algebra,  we  can  show  that 
an  ESS  involving  all  four  pure  strategies  may  exist  if  either  of  the  following  two 
equations  is  true. 


T  +  S-~R<P< 


S  +  T 


and 


P> 


S  +  T 


and 


a/3  T  +  S-2P 
l-aj8  P-S 


<N< 


aj8  R-P 
l-apT-R' 


aj8  R-P 
l-aj8  r-7?’ 
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CHAPTER  4: 
Case  Studies 


In  this  section,  we  present  a  numerical  analysis  of  the  ESS  for  several  case  studies.  For 
each  case  study,  we  choose  the  four  pay-off  values  r,7?,E,S,  as  defined  in  Table  1.1.  Since 
we  obtain  an  equivalent  game  when  shifting  these  pay-off  values  by  the  same  amount,  or 
scaling  these  pay-off  values  by  the  same  positive  constant,  for  any  set  of  we  can 

always  normalize  these  values  so  that  7?  =  1  and  P  =  0.  The  temptation  pay-off  T  >  R  =  I 
specifies  how  much  it  pays  to  cheat,  and  the  pay-off  S  <  P  =  0  specifies  how  much  it  hurts 
to  get  cheated. 


4.1  Case  Study  with  Large  Temptation  and  Large  Sucker’s 
Payoff 

In  this  example,  we  use  the  following  parameters  for  the  pay-off  of  the  basic  PD  game: 

r  =  1.14, 

7?=1, 

P  =  0, 

5=  -0.29. 

The  temptation  pay-off  T  is  substantially  higher  than  R,  and  the  sucker  pay-off  S  is  sub¬ 
stantially  lower  than  P. 

For  these  parameters,  we  use  equations  (3.2)  and  (3.3)  to  determine  the  ESS,  based  on  the 
population  size  N  and  the  compound  discount  factor  a {5.  The  result  is  shown  in  Figure  4. 1 . 
It  may  be  noted,  in  this  case,  that  the  population  in  which  some  type  of  collaborative  ESS 
exists  tends  to  be  small. 

For  example,  consider  a  standard  army  platoon  of  =  42  men  with  a  compound  discount 
factor  of  =  .95.  The  platoon  will  be  in  a  stable  ESS  of  cooperation  based  on  all  the  men 
playing  TFT,  in  the  bottom-right  yellow  region  of  4.1.  However,  should  the  replacement 
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rate  grow,  perhaps  due  to  a  low  retention  rate,  then  /3  deereases.  The  value  of  the  future 
will  then  be  discounted  more  heavily.  As  a/3  <  .934,  a  phase  shift  will  occur,  which  will 
introduce  suspicion  and  defection  into  the  platoon. 

At  this  point,  we  may  expect  that  some  men  will  request  a  transfer  out  of  the  platoon,  as 
the  ESS  is  rather  poor  to  live  with.  As  a  result,  the  future  may  be  further  discounted.  The 
increase  in  /3,  in  turn,  may  bring  an  increase  in  the  proportion  of  men  playing  AD  until 
an  ESS  of  constant  defection  is  reached.  This  case  study  shows  the  importance  of  a  low 
mobility  rate  in  small,  cooperative  groups. 


Equilibrium  Zones  for  T=1 .14,  S=-0.29 


0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9 


Compound  discount  factor  (ap) 


Figure  4.1:  The  potential  ESS  zones  corresponding  to  the  case  of  T=1.14,  R=l,  P=0  and 
S=-0.29.  The  red  area  represents  combinations  of  A  and  aj3,  for  which  the  only  stable  ESS  is 
constant  defection  by  everyone  in  the  population.  The  blue,  next  on  the  way  towards  the  bottom 
right  corner  of  the  chart,  represents  an  area  in  which  TFT  and  AD  are  in  a  mixed  ESS  or  TFT 
exists  as  a  pure  strategy  ESS.  In  addition  to  those  two  states,  the  white  area  is  a  region  in  which 
all  4  strategies  are  an  ESS.  In  the  grey,  a  STFT  and  TF2T  mixed  equilibrium  exists,  as  well.  In 
the  bottom-right  yellow  region  only  a  pure  TFT  or  a  STFT-TF2T  mix  occurs. 
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4.2  Case  Study  with  Small  Temptation  and  Large  Sucker’s 
Payoff 

In  this  example,  we  use  the  following  parameters  for  the  pay-off  of  the  basie  PD  game. 

r  =  1.02, 

i?=l, 

P  =  0, 

5=  -0.29. 

The  temptation  pay-off  T  is  slightly  higher  than  R,  and  the  sucker  pay-off  S  is  substantially 
lower  than  P. 

For  these  parameters,  we  calculate  what  type  of  ESS  are  possible  depending  on  the  pop¬ 
ulation  size  and  on  the  compound  discount  factor  a/3.  The  result  is  shown  in  Figure  4.2. 
It  may  be  noted,  in  this  case,  that  the  population  in  which  some  type  of  collaborative  ESS 
exists  tends  to  be  large. 

For  example,  we  consider  a  town  with  a  population  oi  N  =  500  and  a  discount  factor  of 
aj8  =  .996  existing  in  an  ESS  of  everyone  playing  TFT.  As  long  as  the  town  does  not  grow 
and  the  population  remains  stationary,  there  is  no  reason  for  anyone  in  the  town  to  behave 
in  any  way  other  than  cooperative. 

Should  the  population  of  the  town  ever  increase  past  N  =  640  the  system  may  enter  an 
ESS  with  all  four  strategies  present.  While  some  people  will  continue  to  play  TFT,  some 
people  may  see  the  opportunity  to  play  STFT  or  AD  to  obtain  the  temptation  pay-off  T.  If  a 
substantial  fraction  of  people  play  STFT,  then  some  people  may  become  motivated  to  play 
TF2T  in  order  to  reach  cooperation.  The  presence  of  TF2T  in  the  population  will  also  make 
AD  more  attractive,  for  it  will  produce  two  temptation  pay-offs  at  the  beginning  against  a 
TF2T  player.  If  the  population  continues  to  increase,  the  benefit  of  playing  STFT  and  AD 
will  also  increase.  In  the  end,  everyone  will  play  either  STFT  or  AD  and  no  cooperation  is 
possible. 

An  increase  in  population  mobility,  even  without  changing  population  size,  will  reduce  the 
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value  of  the  future  as  it  is  modeled  by  a  redueed  memory  retention,  eausing  a  deerease 
in  the  value  of  the  future  and  thus  a  ehange  in  ESS.  If  the  value  of  the  discount  factor  is 
reduced  below  .996  the  only  ESS  position  for  the  population  will  be  AD  by  everyone.  If 
that  happens  we  may  expect  the  town  to  be  abandoned  by  many  citizens,  until  at  some  point 
the  population  will  be  reduced  enough,  to  cause  another  phase  change  back  into  the  fold  of 
cooperation. 


Equilibrium  Zones  for  T=1 .02,  S=-0.29 


0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9 


Compound  discount  factor  (ap) 

Figure  4.2:  The  potential  ESS  zones  corresponding  to  the  case  of  T=1.02,  R=l,  P=0  and 
S=-0.29.  The  red  area  represents  combinations  of  A  and  aj3,  for  which  the  only  stable  ESS  is 
constant  defection  by  everyone  in  the  population.  The  blue,  next  on  the  way  towards  the  bottom 
right  corner  of  the  chart,  represents  an  area  in  which  TFT  and  AD  are  in  a  mixed  ESS  or  TFT 
exists  as  a  pure  strategy  ESS.  In  addition  to  those  two  states,  the  white  area  is  a  region  in  which 
all  4  strategies  are  an  ESS.  In  the  grey,  a  STFT  and  TF2T  mixed  equilibrium  exists,  as  well.  In 
the  bottom-right  yellow  region  only  a  pure  TFT  or  a  STFT-TF2T  mix  occurs. 
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4.3  Case  Study  with  Small  Temptation  and  Small  Sucker’s 
Payoff 


In  this  example,  we  use  the  following  parameters  for  the  pay-off  of  the  basic  PD  game. 

r  =  1.02, 

^=1, 

P  =  0, 

5=  -0.01. 

The  temptation  pay-off  T  is  slightly  higher  than  R,  and  the  sucker  pay-off  S  is  slightly  lower 
than  P. 

For  these  parameters,  we  calculate  what  type  of  ESS  are  possible  depending  on  the  popu¬ 
lation  size  and  on  the  compound  discount  factor  aj8.  The  result  is  shown  in  Figure  4.3.  It 
may  be  noted,  in  this  case,  that  the  most  robust  ESS  involves  the  suspicious  strategy,  STFT. 

We  consider  a  start-up  company  of  =  50  employees.  Since  the  future  of  the  enterprise 
is  uncertain  we  use  =  0.7.  The  managers  have  managed  to  raise  the  required  funds 
to  significantly  expand  their  business.  They  may  hire  100  additional  employees  over  the 
course  of  a  short  period  of  time.  As  a  result,  the  population  of  the  company  would  be 
N  =  150,  and  the  unanimous  cooperation  that  gave  the  group  their  edge,  will  become  an 
ESS  of  the  suspicious  strategy  with  TFT.  If  the  team  manages  to  pull  through  the  expansion 
phase,  their  future  may  become  more  certain,  bringing  the  value  of  the  future  parameters 
up.  If  >  0.76  then  cooperation  based  on  pure  TFT  would  once  more  become  the  norm. 
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Compound  discount  factor  (ap) 


Figure  4.3:  The  potential  ESS  zones  corresponding  to  the  case  of  T=1.02,  R=l,  P=0  and 
S=-0.01.  The  red  area  represents  combinations  of  and  aj3,  for  which  the  only  stable  ESS 
is  constant  defection  by  everyone  in  the  population.  The  yellow,  next  on  the  way  towards  the 
bottom  right  corner  of  the  chart,  represents  an  area  in  which  TFT  and  STFT  are  in  a  mixed 
ESS.  In  the  grey  region,  a  STFT  and  TF2T  mixed  equilibrium  exists,  as  well  as  a  pure  strategy 
ESS  of  TFT. 


4.4  Case  Study  with  Large  Temptation  and  Small  Sucker’s 
Payoff 

In  this  example,  we  use  the  following  parameters  for  the  pay-off  of  the  basie  PD  game. 

T  =  1.14, 

R=l, 

P  =  0, 

5=  -0.01. 

The  temptation  pay-off  T  is  substantially  higher  than  R,  and  the  sueker  pay-off  S  is  slightly 
lower  than  P. 

For  these  parameters,  we  ealeulate  what  type  of  ESS  are  possible  depending  on  the  popu¬ 
lation  size  and  on  the  eompound  diseount  faetor  a/3.  With  these  parameters  the  region  of 
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suspicious  ESS  is  clearly  much  larger  than  in  the  previous  seetion,  as  seen  by  a  eomparison 
of  Figure  4.4  and  Figure  4.3. 

We  return  to  eonsider  the  start-up  eompany  from  the  previous  section.  A  group  operat¬ 
ing  under  these  pay-offs  would  expeet  to  beeome  suspieious  at  a  much  earlier  phase  of 
reeruitment.  For  a  value  of  eompound  diseount  faetor  of  a/3  =  0.7  even  a  population  of 
A  >  14  would  begin  to  exhibit  suspicion.  However,  this  set  of  parameters  is  more  robust 
with  respeet  to  eomplete  loss  of  eooperation.  It  ean  be  seen  that  this  situation  may  sup¬ 
port  a  suspieious  eollaboration  even  for  a  group  as  large  as  N  =  250.  This  extension  of 
eooperation  by  use  of  suspicion  may  be  of  advantage  in  certain  situations. 


Equilibrium  Zones  for  T=1 .14,  S=-0.01 


0.1  0.2  0.3  0.4  0.5  0.6  0.7  0.8  0.9 


Compound  discount  factor  (ap) 

Figure  4.4:  The  potential  ESS  zones  corresponding  to  the  case  of  T=1.14,  R=l,  P=0  and 
S=-0.01.  The  red  area  represents  combinations  of  A  and  aj3,  for  which  the  only  stable  ESS 
is  constant  defection  by  everyone  in  the  population.  The  yellow,  next  on  the  way  towards  the 
bottom  right  corner  of  the  chart,  represents  an  area  in  which  TFT  and  STFT  are  in  a  mixed 
ESS.  In  the  grey  region,  a  STFT  and  TF2T  mixed  equilibrium  exists,  as  well  as  a  pure  strategy 
ESS  of  TFT. 
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CHAPTER  5: 
Conclusion 


Research  has  shown  that  cooperation  is  constrained  with  relation  to  group  size.  In  this  the¬ 
sis,  we  present  a  model  for  interaction  in  a  finite  population  of  size  N,  engaged  in  iterative 
rounds  of  prisoner’s  dilemma.  The  model  allows  us  to  explore  conditions  for  the  viability 
of  cooperative  equilibrium.  We  are  also  able  to  explain  the  importance  of  suspicion,  as  a 
form  of  behavior  that  inhibits  uncooperative  behavior,  under  certain  conditions.  Among  the 
factors  used  to  explain  conditions  for  cooperation,  a  model  for  memory  in  the  population 
is  presented,  as  a  geometric  random  variable,  which  can  also  be  used  to  model  mobility  of 
individuals  in  the  population. 


Several  strategies  are  used  by  the  population  in  the  model.  Tit-for-two-tats,  tit-for-tat, 
suspicious-tit-for-tat  and  constant  defection  allow  a  study  of  some  social  phenomenon. 
There  are  15  different  possible  combinations  of  strategies,  each  involving  either  one,  two, 
three  or  four  pure  strategies,  that  may  be  prevalent  in  the  population.  We  find  the  different 
necessary  conditions  for  combinations  of  strategies  to  become  equilibrium  states,  using  for 
this  purpose  the  concept  of  an  evolutionarily  stable  strategy  (ESS). 


For  any  parameters  of  the  game  an  upper  limit  on  population  size  was  found,  above  which, 
the  only  ESS  is  constant  defection  by  everyone  in  the  population.  All  the  different  con¬ 
ditions  found  for  a  population  size,  that  allow  a  collaborative  ESS,  include  a  term  that 
depends  on  the  compound  value  of  the  future, 


N 


aj5 

l-aj5' 


This  shows  that  increased  population  mobility  results  in  significant  lowering  of  viability  of 
cooperation.  The  different  conditions  on  population,  also  depend  on  a  ratio  of  differences, 
of  the  pay-offs,  of  the  single  round  prisoner’s  dilemma.  For  some  sets  of  parameter,  the 
collaborative  ESS,  that  remains  viable  in  the  largest  population,  employs  the  suspicious 
STFT  strategy.  This  justifies  the  existence  of  suspicion  in  observed  behavior,  as  a  means  to 
ensure  cooperation  remains  viable,  despite  temptation  to  defect. 
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In  this  thesis,  several  assumptions  may  not  be  entirely  realistic.  First,  each  player  has  to 
pick  a  strategy,  which  he  uses  for  the  whole  game,  from  a  set  of  four  options,  whereas 
in  real-life  people  may  be  more  creative  in  choosing  a  strategy.  Second,  we  use  the  geo¬ 
metric  distribution  to  model  an  individual’s  memory,  and  also  for  the  duration  of  time  an 
individual  stays  in  the  population  before  moving  out.  Third,  we  assume  that  the  popula¬ 
tion  is  homogeneous;  that  is,  every  individual  has  the  same  value  functions.  In  real-life, 
however,  the  pay-off  for  cooperation  may  be  higher  for  some  people  than  for  some  other 
people.  Fourth,  we  assume  that  each  individual  will  encounter  every  other  individual  in  the 
population  with  the  same  probability,  whereas  in  real-life  neighbors  may  encounter  more 
frequently.  Relaxing  these  assumptions  presents  challenging  future  research  directions. 
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